{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# EarthNet2021 Model Intercomparson Suite Cookbook\n",
    "\n",
    "Run.py is the main entry point. You should be able to launch any task from it! But first let's...\n",
    "\n",
    "## Download EarthNet2021 Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import earthnet as en\n",
    "\n",
    "en.Downloader.get('data/datasets/release', 'all') #[\"train\",\"iid\",\"ood\",\"extreme\",\"seasonal\"]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test Baseline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!python run.py --model_name Baseline --split_name iid_test_split --submodel mean"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train\n",
    "The EN Intercomparison suite comes with a Tensorflow baseline based on SAVP that we call Arcon-STF (Autorgressive Conditional Spatio-Temporal Forecaster).\n",
    "You can train and test any model using run.py. Checkout 'configs/tf_template/' to learn how the standalone model is passed arguments from run.py."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Firt we generate tfrecords samples for the train set using the included util\n",
    "!source activate ENtf115py36; python utils/npz_to_tfecords.py --inpath data/datasets/release/train/ --outpath data/tf_dataset/train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Running train Arcon trough run.py\n",
    "\n",
    "!python run.py --model_name tf_template --dataroot data/tf_dataset/ --split_name train --experiment_name sample_hparams_Arcon-STF --mode train --submodel arcon_earthnet --dataset earthnet --gpu_ids 0 --experiment_settings sample_hparams_Arcon-STF"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#To continue training a paused train run\n",
    "\n",
    "!python run.py --model_name tf_template --dataroot data/tf_dataset/ --split_name train --experiment_name sample_hparams_Arcon-STF --mode train --submodel arcon_earthnet --dataset earthnet --gpu_ids 0 --resume"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# We generate tfrecords samples for the iid_test set using the included util\n",
    "!source activate ENtf115py36; python utils/npz_to_tfecords.py --inpath data/datasets/release/iid_test_split/ --outpath data/tf_dataset/iid_test_split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Predicting tf_template's Arcon model\n",
    "\n",
    "!python run.py --model_name tf_template --dataroot data/tf_dataset/ --split_name iid --experiment_name sample_hparams_Arcon-STF --mode test --submodel arcon_earthnet --dataset earthnet --gpu_ids 0 --num_stochastic_samples 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Predicting Baseline model linear regression\n",
    "\n",
    "!python run.py --model_name Baseline --split_name iid_test_split --submodel mean"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Evaluates the output of the model Baseline\n",
    "!python run.py --mode evaluate --model_name Baseline --experiment_name mean"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Evaluates the output of the tf_template Arcon\n",
    "!python run.py --mode evaluate --model_name tf_template --split_name iid_test_split --experiment_name sample_hparams_Arcon-STF "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#plotting the figures to display evaluation results\n",
    "# use --animations to create videos of [0,0.15,0.3,0.5,0.7,0.85,1] quantile samples by model performance.\n",
    "!python run.py --mode plot\n",
    "#Find them in data/results/figures"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plot\n",
    "Quick check on how to use the utility to plot multicubes in utils/plot_sample.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Plotting a single frame from NPZ with predictions\n",
    "\n",
    "!python utils/plot_sample.py --pred --frame 150 --image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Plotting a .mp4 video\n",
    "\n",
    "!python utils/plot_sample.py --pred --frame 150 --video --tilename 29SND --filename 29SND_2017-06-10_2017-11-06_2105_2233_2873_3001_32_112_44_124.npz --datapath data/datasets/release/train\n",
    "\n",
    "#Plotting samples on the test sets (whose groundtruth is split into context and target files is not yet working)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
